Deep Cross-media Knowledge Transfer
نویسندگان
چکیده
Cross-media retrieval is a research hotspot in multimedia area, which aims to perform retrieval across different media types such as image and text. The performance of existing methods usually relies on labeled data for model training. However, cross-media data is very labor consuming to collect and label, so how to transfer valuable knowledge in existing data to new data is a key problem towards application. For achieving the goal, this paper proposes deep cross-media knowledge transfer (DCKT) approach, which transfers knowledge from a large-scale cross-media dataset to promote the model training on another smallscale cross-media dataset. The main contributions of DCKT are: (1) Two-level transfer architecture is proposed to jointly minimize the media-level and correlation-level domain discrepancies, which allows two important and complementary aspects of knowledge to be transferred: intramedia semantic and inter-media correlation knowledge. It can enrich the training information and boost the retrieval accuracy. (2) Progressive transfer mechanism is proposed to iteratively select training samples with ascending transfer difficulties, via the metric of cross-media domain consistency with adaptive feedback. It can drive the transfer process to gradually reduce vast cross-media domain discrepancy, so as to enhance the robustness of model training. For verifying the effectiveness of DCKT, we take the largescale dataset XMediaNet as source domain, and 3 widelyused datasets as target domain for cross-media retrieval. Experimental results show that DCKT achieves promising improvement on retrieval accuracy.
منابع مشابه
Comparative Analysis of Information Dissemination Capabilities of Media and Social Networks
Background and Aim: Human Knowledge depends on data and information that is emerged and transffered from different channels. The dessimination process is different from type, form of transfer, and distribution based on information or awareness. This survey compares the librarians and information scienctist’s information transferring capabilities in mass media and social networks. Methods: This ...
متن کاملSemi-Supervised and Cross-Lingual Knowledge Transfer Learnings for DNN Hybrid Acoustic Models Under Low-Resource Conditions
Semi-supervised and cross-lingual knowledge transfer learnings are two strategies for boosting performance of lowresource speech recognition systems. In this paper, we propose a unified knowledge transfer learning method to deal with these two learning tasks. Such a knowledge transfer learning is realized by fine-tuning of Deep Neural Network (DNN). We demonstrate its effectiveness in both mono...
متن کاملTransfer and Structure Learning in Markov Logic Networks
Markov logic networks are a recently developed knowledge representation capable of compactly representing complex relationships and handling uncertainty in a principled manner. The deep transfer algorithm of Davis and Domingos proposes a method for learning the structure of an MLN by incorporating cross-domain knowledge — for example, using the relationships between yeast proteins to inform pre...
متن کاملLearning to Cross Boundaries in Online Knowledge Communities: Fading of Surface-level and Rise of Deep-level Similarity with Experience
Many organizations have launched online knowledge forums to promote knowledge flow across boundaries. This paper theorizes and empirically tests whether employees transfer knowledge within or across boundaries and how the tendencies change as a function of a knowledge provider’s experience in an online forum. We suggest that participants prefer to transfer knowledge to others with whom they sha...
متن کاملMHTN: Modal-adversarial Hybrid Transfer Network for Cross-modal Retrieval
Cross-modal retrieval has drawn wide interest for retrieval across different modalities of data (such as text, image, video, audio and 3D model). However, existing methods based on deep neural network (DNN) often face the challenge of insufficient cross-modal training data, which limits the training effectiveness and easily leads to overfitting. Transfer learning is usually adopted for relievin...
متن کامل